The development and implementation of a new hospital performance measure to assess hospital contributions to community health and equity

Abstract Objective To develop and implement a measure of how US hospitals contribute to community health with a focus on equity. Data Sources Primary data from public comments and hospital surveys and secondary data from the IBM Watson Top 100 Hospitals program collected in the United States in 2020 and 2021. Study Design A thematic analysis of public comments on the proposed measure was conducted using an iterative grounded approach for theme identification. A cross‐sectional survey of 207 hospitals was conducted to assess self‐attestation to 28 community health best practice standards in the revised measure. An analysis of hospital rankings before and after inclusion of the new measure was performed. Data Collection/Extraction Methods Public comment on the proposed measure was collected via an online survey, email, and virtual meetings in 2020. The survey of hospitals was conducted online by IBM in 2021. The analysis of hospital ranking compared the 2020 and 2021 IBM Watson Top 100 Hospitals program results. Principal Findings More than 650 discrete comments from 83 stakeholders were received and analyzed during measure development. Key themes identified in thematic analysis included equity, fairness, and community priorities. Hospitals that responded to a cross‐sectional survey reported meeting on average 76% of applicable best practice standards. Least met standards included providing emergent buprenorphine treatment for opioid use disorder (53%), supporting an evidence‐based home visiting program (53%), and establishing a returning citizens employment program (27%). Thirty‐seven hospitals shifted position in the 100 Top Hospital rankings after the inclusion of the new measure. Conclusions There is broad interest in measuring hospital contributions to community health with a focus on equity. Many highly ranked hospitals report meeting best practice standards, but significant gaps remain. Improving measurement to incentivize greater hospital contributions to community health and equity is an important priority.

practice standards, but significant gaps remain. Improving measurement to incentivize greater hospital contributions to community health and equity is an important priority.

K E Y W O R D S
community health, health care reporting, health equity, hospital quality, measures, methods

What is known on this topic
• There is growing appreciation of the unique role of hospitals in addressing health disparities by improving the health of their surrounding communities.
• Investments in improving community health and in addressing social determinants complement efforts to provide quality and equitable care.
• No leading hospital evaluation or ranking programs have incorporated a set of community health measures prior to this work.

What this study adds
• The development and implementation of a new survey-based instrument to measure how hospitals contribute to community health, with a focus on equity, drew broad interest, and engagement.
• Implementation of the new measure demonstrated that a majority of top hospitals are engaging in activities that promote community health, but there remain significant opportunities for further progress.
• There is a need for further development of tools to assess the impact of hospital contributions to community health and health equity.

| INTRODUCTION
Over the last decade, the United States has experienced both escalating health care costs and a plateau and decline in life expectancy, with the persistence of significant racial and socioeconomic disparities in health and health care. [1][2][3][4] More than a century of research demonstrates that social and economic conditions in communities where people live, work, learn, and play are key drivers of population-level health outcomes and, by extension, health disparities. 5,6 Recognizing this dilemma, hospitals are increasingly looking beyond their walls to make contributions to population health and health equity. [7][8][9][10] Hospital ranking and recognition programs are early in the process of adjusting to these changes. The American Hospital Association's Foster McGaw Prize uses a non-standardized approach to recognize hospitals for community service. 7 The Lown Index aims to rank hospitals by their social responsibility using publicly available data related to civic leadership, value of care, and community benefits. 8 Leading hospital performance evaluation methodologies, including US News and World Report and Leapfrog, however, have remained largely focused on patient outcomes, finances, and patient experience. [9][10][11] The IBM Watson Health 100 Top Hospitals study (hereto referred as 100 Top study) has assessed US hospitals for the last 28 years. This national quantitative study evaluates short-term, nonfederal, acute care hospitals by utilizing publicly available data to assess hospital performance in five domains: inpatient outcomes, extended outcomes, operational efficiency, financial health, and patient experience. 9 In the second stage of measure development, the Johns Hopkins team sought public comment on the draft proposal. Invitations to comment were sent to state hospital associations; experts in the fields of medicine, hospital administration, public health, and health care and social policy; and to community-based organizations, health care organizations, and nonprofits.
Comments on the initial proposal were submitted through (1) an online survey (Survey Monkey), which prompted participants to rank their agreement with inclusion of each best practice using a Likert scale (1 = strongly disagree to 5 = strongly agree) and also to include free-text comments about the proposed best practices; (2) email sub- In the third stage of measure development, the Johns Hopkins team used the public comments to revise the draft measure. The team grouped all of the discrete comments received into key themes and considered additional proposed best practices submitted by stakeholders. All stakeholder feedback (i.e., survey responses, emailed comments, and contemporaneous notes from webinars and meetings) was loaded into a database (Knack, version 3) and tagged by topic area.
The comments were sorted by topic and grouped according to thematic content. In addition to identifying critiques or recommendations specific to proposed best practices, the analytic process identified cross-cutting themes or recurring related ideas that applied to all or multiple domains of the proposed measure. The team arrived at a consensus around theme identification through an iterative process.  institution. The survey also included the option for hospitals without obstetrics facilities to mark obstetrics specific questions as not applicable and to provide links to relevant websites for more information.

| Survey of hospitals
For hospitals that were unable to be contacted by email, the administrators for the 100 Top study attempted contact by phone.
Survey data were analyzed using Stata. 15 The analysis included the number of best practice standards in each domain attested to by each hospital and whether there were significant differences across hospitals by the following hospital characteristics: region of the country; type of community served (i.e., urban vs. rural); academic status; health system status; peer group (major teaching hospital, teaching hospital, small community, medium community, large community); ownership (governmental, proprietary, nonprofit); and payer mix.
Payer mix reflected the percent of hospital days paid for by Medicare and Medicaid based on a binary variable of whether the hospital exceeded the median value for each (median Medicare payer percent was 30.7% and median Medicaid payer percent was 6.6%).
Exploratory analyses using chi-square tests examined whether there were statistically significant differences in reported best practices among responding hospitals across each of the hospital characteristics described. Statistical significance was determined using a p-value significance of less than 0.05. Some respondents commented that more attention should be paid to racial justice considerations across the proposed metrics. One comment stated, "As applicable, all of these standards should seek to assess relevant equity gaps and measure closure of these gaps by race/ethnicity and income."

| Measure implementation
Other comments pointed out challenges related to measuring disparities between groups directly, including the availability and reliability of data and the difficulty in identifying measures that work across different types of hospitals and communities. For example, some comments noted that the assessment of disparities may be particularly challenging in communities where the population is less racially diverse.
In response to these comments, the Johns Hopkins team supported the further development and adoption of metrics and standards that could increase accountability for reducing disparities in health and health care, including through the use of explicit measures of racial equity. Fundamentally, an equity-driven approach requires attention to populations of greatest need, including historically marginalized populations.  (4) Hospital Association 6% (4) School of Public Health 4% (3) Dept of Health 3% (2) Community organization 2% (1) Health data services 2% (1) Health plan 2% (1) Individual 2% (1) Specialty society 2% (1) N/A 12% (8) Total 100% (67) Note: Organizational affinities of responders to the online survey during the public comment period of the measure development. These affinities are self-identified, and not all commenters provided an affinity (N/A).

| Fairness
Some respondents raised concerns about the fairness of the proposed measure. For example, one stated, "I think it's unfair for health systems and hospitals alone to solve societal problems." Another comment stated that the "amount a health system can contribute to the success of these metrics may vary depending on a number of factors, including resources." Other fairness concerns included questioning whether it was appropriate to apply one set of standards to the entire hospital industry.
At the same time, many respondents noted ways in which hospitals can contribute to community health, with one comment recognizing that it is important for hospitals to "be accountable for their communities…beyond health fairs and covering charity care." Another stated, "We appreciate the basic concept of the frameworkthat is, to reflect the enormous efforts that hospitals and health systems undertake to contribute to the health and well-being of their communities, and to reduce health care disparities."

| Community priorities
Some respondents called for more attention to self-identified community needs, with one saying, "We do not take issue with the standards proposed; however, there is risk in prescribing standards without knowing the unique needs of the community in which the standards would be applied." In response to this comment, the Johns Hopkins team noted it would not be feasible for a ranking system to assess community activities of individual hospitals, ascertain the priorities of each community, and compare the two. It was noted that the community health issues covered by the best practice standards are broadly relevant to diverse US communities, and meeting all the standards proposed would not be required for full credit. In addition, the Johns Hopkins team modified the proposed best practice standard on community health needs assessment to require the input of communities in this process.  The hospitals reported meeting an average of 76% of applicable best practice standards across the three measure domains (i.e. hospital as [1] health care provider, [2] community partner, and [3] anchor institution). Ninety (78%) met at least half of applicable best practice standards in all three areas, 16 (14%) in two areas, seven (6%) in one area, and three (3%) in no areas (Table 3).

| Hospitals as health care providers
Hospitals met an average of 9.6 (range: 2-12) of the applicable best practice standards in this area (12 maximum Hospitals that are part of a health system more often reported supporting an evidence-based home visiting program. The standards that were most frequently reported as met were having a community needs assessment (93%, 108 hospitals) and supporting school success (91%, 105 hospitals) ( Table 4). Northeast-located hospitals were less likely to report having a plan for advancing sustainability. West-located and nonprofit-owned hospitals were less likely to report having a living hourly wage.

| Hospitals as anchor institutions
The standard most frequently reported as met was offering paid sick leave to all employees (91%, 105 hospitals) ( Table 4).

| DISCUSSION
The development and implementation of a measure of hospital contributions to community health, with a focus on equity, generated substantial interest and participation. Worsening health statistics, Note: Characteristics of the top 200 hospitals invited to respond to the hospital measure survey, by responder status. Data reported as the percent (n) of hospitals meeting the criteria in column one out of the 116 hospitals that returned a survey (column 2) or out of the 91 hospitals that did not return the survey (column 3). p < 0.05 was used to determine statistical significance.
T A B L E 4 Association of hospital characteristics with best practice standard attestation continued health inequities, and rising costs underscore the importance of this project. The substantial participation of health systems in the discussion reflects greater appreciation by leadership of the potential for hospitals to make a difference through efforts and partnerships that extend beyond the hospital walls.
How best to measure these contributions, however, is unsettled.
Publicly available data are neither specific to local health challenges nor to the potential steps that hospitals can take. At the same time, the range of potential hospital activities that might be linked to community health is quite broad. The IBM Watson measure was devel- Relatively few hospitals attested to supporting an evidence-based home visiting program (53%, 61 hospitals), despite extensive and consistent evidence of the positive impact of such programs on maternal and child health outcomes, including improved cognitive and behavioral outcomes, lower rates of maternal mortality, and reduced low birth weight. 21,22 Further, just over a quarter of hospitals reported having a returning citizens employment program, despite such programs providing more employment opportunities for previously incarcerated individuals, who are disproportionately people of color, and who face more barriers when seeking employment. 23,24 One way to encourage the adoption of these and other similar practices is to appeal to the social mission of hospitals, most of which are critical community institutions with deep roots in the regions they serve. Another approach is to align financial incentives with community health outcomes, a key goal of population-based payment reform. A total of 116 hospitals were eligible for each best practice standard, with two exceptions. One hundred two hospitals were eligible for the "supports breastfeeding" standard, and 93 were eligible for "contraception treatment and counseling post-partum." lack of a formal verification system, and selection bias. Hospital selfreport surveys are used in several existing hospital assessment tools, including Leapfrog Rating and US News Best Children's Hospital rankings. 25, 26 Future iterations of the measure may include more robust methods for increasing response rate and holding hospitals accountable for the best practices that they attest to.
The study of current hospital community health practices is also limited by the fact that only the top 200 hospitals were surveyed during the measure implementation's pilot year. It is therefore not clear the extent to which these results are representative of hospitals across the country.
Despite these limitations, this initial analysis demonstrated that measurement of hospital contributions to community health and equity is possible. It also revealed four major areas for future work.
First, progress on measurement is urgently needed. One of the challenges highlighted in the development of this measure is the lack of readily available population health outcomes by race, ethnicity, or socioeconomic status at the community level. Furthermore, most national data sources are not available at the sub-county level for health outcomes, which limits the ability of hospitals to look at impacts on their immediate geography.
Although outside the scope of this measure, the comment period also revealed tremendous interest in more comprehensive assessment of equity in the delivery of health services. There is no national source of clinical outcomes by race or ethnicity, such as readmission rates, adverse outcomes, or patient experience. Attention to these gaps should be a priority.
Second, there should be more research on the impact of specific hospital practices on community health. Many of the best practice standards included in the new measure relate both directly and indirectly to community health outcomes, but studies should assess whether hospital contributions demonstrably impact community health at scale. A curation of impact evaluations within the peer-reviewed literature would raise certain best practices above the others. To date, the majority of research, societal guidelines, and consensus documents have focused on clinical interventions by hospitals. Evidence assessing how hospitals can positively impact community health and advance health equity in their roles as community partners or anchor institutions should be further developed.
Third, more attention is due at the interface of hospital practice and public policy. Hospitals have financial, environmental, and political impacts within their communities. Assessing such impacts should be considered for future measures of this type.
Fourth, measurements of hospital efforts should expand-beyond these initial ideas, beyond 200 high-performing hospitals, and beyond one ranking system. Given the urgency of improving health outcomes in the United States, no comprehensive hospital performance evaluation should be complete without addressing contributions to community health with a focus on equity.

| CONCLUSIONS
Broad stakeholder participation led to the creation of a hospital measure to assess contributions to community health with a focus on equity. Most leading hospitals self-attest to many, but not all, of the 28 best practice standards incorporated in the measure. Inclusion of the measure in the 2021 100 Top study led to a substantial shift in hospital rankings. Future work should expand and improve upon this initial effort.